STAT3926/STAT4026: Statistical Consulting

Forecasting Model for Study Load at the University of Sydney

Authors

Prepared by: 520432255, 510288769, 510456265

Prepared for: Susie Chee

Published

May 20, 2024

Executive Summary

Following recent policy changes regarding International Student enrolment requirements, the client requires a forecasting model to estimate student enrolments and confirm the University’s budget before the Census date. A linear model was created for each faculty, considering the different fee types, years and semesters and the COVID-19 impact from 2020 to 2021. Based on the 2024 linear model predictions, the University is expected to reach budget. However, training the linear model with additional historical data (including consideration of additional variables) is recommended to improve the model’s accuracy.

Background

In 2024, the Australian government implemented policy changes for international students at tertiary education institutions designed to enhance the quality of education and reinforce the integrity of the visa system. A key aspect of these changes is that international fee-paying students are no longer mandated to enrol for both Semester 1 and Semester 2 during the initial enrollment period of Semester 1. This shift moves away from the traditional requirement for international students to commit to a full academic year upfront.

This policy adjustment will influence how international students approach their education in Australia. This change may result in more fluctuating enrollment figures from one semester to the next, posing a challenge for the University of Sydney regarding planning and resource allocation.

Glossary

CSP (Commonwealth Supported Places): A funding scheme where the Australian government subsidises the tuition fee of students at Australian universities. This support is only available to domestic students.

DFEE (Domestic Fee-Paying Students): Students who do not qualify for Commonwealth support and must pay full tuition fees. These fees are typically higher than those for Commonwealth Supported Places.

IFEE (International Fee-Paying Students): Students from countries other than Australia who attend Australian universities. These students pay full tuition fees, which are usually higher than the fees for domestic students.

EFTSL (Equivalent Full-Time Student Load): A measure used to indicate the standard annual full-time study load. One EFTSL is typically equivalent to one year of full-time study. This measure is used to calculate the study load of part-time students in proportion to a full-time study load.

Client’s Aims

The client requires a forecasting model that can be employed early in the academic year—specifically, pre-census in January or February. This model is intended to provide a reliable preliminary estimate of the entire year’s enrolment figures. It must also adapt to recent policy changes that have eliminated the requirement for international students to enrol for both semesters at the start of the academic year.

Data

The provided data, including EFTSL values for 2018-2023, has been plotted below. Most faculties generally show a relatively stable trend in EFTSL over this period. In the Faculty of Engineering and Business School, EFTSL is significantly higher for international students. For Sydney Law School, Faculty of Science, Sydney Conservatorium of Music, and Faculty of Medicine and Health, CSP students account for the majority of EFTSL. The Faculty of Arts and Social Sciences has seen IFEE EFTSL overtake CSP since midway through 2020.

Figure 1: EFTSL Across Major USYD Faculties, by Fee Type

Forecast Modelling

A linear model has been created to forecast equivalent full-time study load (EFTSL) for each major Faculty (Faculty of Arts and Social Sciences, Business School, Faculty of Engineering, Faculty of Medicine and Health, Faculty of Science, Sydney School of Architecture, Design and Planning, Sydney Law School and Sydney Conservatorium of Music) within the University. Predicting EFTSL allows for flexibility of the model with variance of University Fees for each faculty and accounts for the varied study load of each student. For each linear model, Fee Type, Year and Semester are considered. A dummy variable was also added to consider the impact of the COVID-19 pandemic on EFTSL.

Assumptions of linear modelling include linearity, homoscedasticity, independence, and normality. Given that each of the data points is independent due to being separated across Semester, Year, Fee Type, and Faculty and has a linear relationship, these assumptions have been met. Further diagnostic plots to demonstrate this can be found in the Appendix (Section 7).

A general summary for each model is tabulated below in Table 1, and the coefficients of each model are in Table 6. A model was created for each combination of faculty and fee types, resulting in a total of 24 models.

Table 1:

Model Summary
Faculty of Teaching Fee Type R-Squared Adjusted R-Squared P-Value AIC Corrected AIC
Business School CSP 0.92 0.89 0.00 87.30 97.30
Business School DFEE 0.05 −0.31 0.94 94.27 104.27
Business School IFEE 0.34 0.09 0.32 150.58 160.58
Faculty of Arts and Social Sciences CSP 0.91 0.88 0.00 110.24 120.24
Faculty of Arts and Social Sciences DFEE 0.43 0.21 0.19 87.66 97.66
Faculty of Arts and Social Sciences IFEE 0.88 0.84 0.00 141.76 151.76
Faculty of Engineering CSP 0.89 0.85 0.00 88.41 98.41
Faculty of Engineering DFEE 0.55 0.39 0.08 62.19 72.19
Faculty of Engineering IFEE 0.90 0.86 0.00 120.59 130.59
Faculty of Medicine and Health CSP 0.35 0.10 0.31 115.49 125.49
Faculty of Medicine and Health DFEE 0.45 0.24 0.17 101.17 111.17
Faculty of Medicine and Health IFEE 0.10 −0.24 0.83 102.21 112.21
Faculty of Science CSP 0.88 0.84 0.00 102.56 112.56
Faculty of Science DFEE 0.81 0.74 0.00 53.62 63.62
Faculty of Science IFEE 0.94 0.91 0.00 108.14 118.14
Sydney Conservatorium of Music CSP 0.88 0.83 0.00 76.28 86.28
Sydney Conservatorium of Music DFEE 0.84 0.78 0.00 20.61 30.61
Sydney Conservatorium of Music IFEE 0.89 0.85 0.00 47.14 57.14
Sydney Law School CSP 0.59 0.43 0.06 82.64 92.64
Sydney Law School DFEE 0.76 0.67 0.01 70.57 80.57
Sydney Law School IFEE 0.92 0.90 0.00 73.92 83.92
Sydney School of Architecture, Design and Planning CSP 0.96 0.95 0.00 63.97 73.97
Sydney School of Architecture, Design and Planning DFEE 0.25 −0.03 0.49 61.54 71.54
Sydney School of Architecture, Design and Planning IFEE 0.96 0.95 0.00 86.96 96.96
Understanding Model Metrics

R-Squared: This value tells us how well our model’s predictions match the actual data. A higher R-Squared value means a better fit.

Adjusted R-Squared: This is a tweaked version of R-Squared that adjusts for the number of predictors used in the model. It provides a more accurate score by considering the complexity of the model. Like R-Squared, a higher value indicates a better fit, but it is generally more reliable especially when comparing models with different numbers of predictors.

P-Value: This number helps us determine whether the results of our analysis are statistically significant. In simple terms, it tests the probability that the relationships observed in your data occurred by chance. A smaller P-Value (typically less than 0.05) suggests that the findings are significant and not just a random occurrence.

AIC (Akaike Information Criterion): This is a tool we use to compare different models. It balances the model’s complexity against how well it fits the data. A lower AIC value indicates a model that better fits the data without becoming overly complex.

Corrected AIC (AICc): This is a version of AIC adjusted for smaller sample sizes. It’s particularly useful when you have a large number of parameters relative to the number of observations. Like AIC, a lower AICc value suggests a better model.

Estimates For Each Coefficients

The chart below represents how various factors affect university fees. It breaks down the influence of different variables like the year, the second semester, and COVID-19 for each department.

Semester 1 was kept as a baseline. This means that when predictions are made for the first semester, the model will only consider the year and the impact of COVID-19. This method helps us focus on what changes, rather than what stays the same.

Similarly, for the COVID-19 impact variable, the default is that there is no COVID-19 impact. This means that when predictions are made for future years, the model doesn’t include it in it’s forecast.

Figure 2: Bar Chart of estimates for CSP across Faculties.


For CSP, EFTSL is lower in Semester 2 across all Faculties. The Business School and Faculty of Science have the largest decrease in EFTSL in Semester 2, while compared to other Faculties this negative impact is minimal in the Sydney School of Architecture, Design and Planning. With every increase in Year, EFTSL is increasing in the Faculty of Medicine and Health, Engineering, the Sydney School of Architecture, Design and Planning and Sydney Conservatorium of Music. EFTSL decreases most every year in the Faculty of Arts and Social Sciences. COVID-19 increased EFTSL in all Faculties except for the Faculty of Medicine and Health, where it had a negative impact.

Figure 3: Bar Chart of estimates for DFEE across Faculties.


For DFEE, EFTSL is lower in Semester 2 across all Faculties except the Business School. The Faculty of Medicine and Health has the largest decrease in EFTSL in Semester 2. As the Year increases, EFTSL increases in the Faculty of Medicine and Health, Business School and Sydney School of Architecture, Design and Planning. The Faculty of Arts and Social Sciences and Sydney Law School have the largest decrease in EFTSL with every increase in Year. COVID-19 increased EFTSL in all Faculties except the Sydney Conservatorium of Music.

Figure 4: Bar Chart of estimates for IFEE across Faculties.


For IFEE, EFTSL is higher in Semester 2 across all Faculties with the exception of Faculty of Medicine and Health and Sydney Conservatorium of Music (by a very small amount). The largest increases in Semester 2 EFTSL occur in the Faculty of Arts and Social Sciences, Business School and Faculty of Engineering. With increases in Year, EFTSL increases in all Faculties, by the highest amount in the Faculty of Arts and Social Sciences, followed by the Faculty of Engineering and Faculty of Science. COVID-19 had a positive impact on EFTSL in the Business School, Faculty of Arts and Social Sciences and Faculty of Science.

Backcasting

Using the linear model, it is possible to compare predictions of EFTSL for previous years, which can be compared with data to demonstrate the model’s performance.

Figure 5: Line Graph of Backcast for Business School across Fee Types.


Figure 6: Line Graph of Backcast for Faculty of Arts and Social Sciences across Fee Types.


Figure 7: Line Graph of Backcast for Faculty of Engineering across Fee Types.


Figure 8: Line Graph of Backcast for Faculty of Medicine and Health across Fee Types.


Figure 9: Line Graph of Backcast for Faculty of Science across Fee Types.


Figure 10: Line Graph of Backcast for Sydney Conservatorium of Music across Fee Types.


Figure 11: Line Graph of Backcast for Sydney Law School across Fee Types.


Figure 12: Line Graph of Backcast for Sydney School of Architecture, Design and Planning across Fee Types.


Forecasting

The forecasting plots below demonstrate EFTSL forecasts for the next five years. Evidently, from the graphs, the model tends to make a large jump in EFTSL estimations. This is likely due to the extrapolation (estimating for larger values than what the data is trained on) occurring here.

Figure 13: Line Graph of Forecast for Business School across Fee Types.


Figure 14: Line Graph of Forecast for Faculty of Arts and Social Sciences across Fee Types.


Figure 15: Line Graph of Forecast for Faculty of Engineering across Fee Types.


Figure 16: Line Graph of Forecast for Faculty of Medicine and Health across Fee Types.


Figure 17: Line Graph of Forecast for Faculty of Science across Fee Types.


Figure 18: Line Graph of Forecast for Sydney Conservatorium of Music across Fee Types.


Figure 19: Line Graph of Forecast for Sydney Law School across Fee Types.


Figure 20: Line Graph of Forecast for Sydney School of Architecture, Design and Planning across Fee Types.


Performance

Given the relatively small amount of training data (5 years) from which the model has been created, the model is somewhat limited. In particular, the model has been overfitted, meaning that the model cannot accurately extrapolate (estimate values for Years outside of the range of Years on which it was trained) and will continue to predict trends that match those in the training data. Due to the nature of the problem, which involves a change in trends of behaviour concerning Semester 2 enrolments there may be additional changes that the model is not accounting for.

Mean Absolute Error (MAE)

We used a method called Mean Absolute Error (MAE) to evaluate how well our forecasting model performed. This method measures the accuracy of our predictions by calculating the average difference between the predicted student enrollments (EFTSL) and the actual enrollments recorded in 2023. The differences are taken as absolute values, which means we focus only on the size of the errors without considering whether the predictions were too high or too low.

Our results showed that the MAE for our model was higher than the MAE for the existing forecasts. This suggests that our model’s predictions were generally further from the actual numbers than the existing forecasts. The larger MAE indicates that our model was less accurate, meaning our predictions about student enrollment were not as close to the actual figures as those of the existing forecasts.

This is likely due to the small size (\(n = 5\) years) of the data from which the model was built, as well as our model’s inability to account for additional factors of variability in EFTSL compared to the existing model.

Table 2: Mean Absolute Error (MAE) for Linear Model compared to Existing Forecast
Faculty of teaching Fee Type Linear Model MAE Existing Forecast MAE
Business School CSP 20.93 26.00
Business School DFEE 28.76 6.06
Business School IFEE 256.36 475.57
Faculty of Arts and Social Sciences CSP 51.31 32.52
Faculty of Arts and Social Sciences DFEE 21.13 4.62
Faculty of Arts and Social Sciences IFEE 184.89 72.13
Faculty of Engineering CSP 21.70 14.38
Faculty of Engineering DFEE 7.00 3.59
Faculty of Engineering IFEE 81.17 27.55
Faculty of Medicine and Health CSP 69.36 15.18
Faculty of Medicine and Health DFEE 38.67 10.01
Faculty of Medicine and Health IFEE 43.59 8.80
Faculty of Science CSP 39.26 18.61
Faculty of Science DFEE 5.29 2.28
Faculty of Science IFEE 44.57 15.91
Sydney Conservatorium of Music CSP 12.77 7.18
Sydney Conservatorium of Music DFEE 1.29 0.32
Sydney Conservatorium of Music IFEE 3.43 1.29
Sydney Law School CSP 18.15 8.68
Sydney Law School DFEE 9.70 6.28
Sydney Law School IFEE 12.24 6.76
Sydney School of Architecture, Design and Planning CSP 7.71 3.58
Sydney School of Architecture, Design and Planning DFEE 6.86 1.21
Sydney School of Architecture, Design and Planning IFEE 21.11 3.90

Percentage Change

Percentage change was also investigated to compare the existing forecast and the linear model. Examining differences in percentage change shows that the model performs most accurately with the CSP and IFEE types. There is a large variance in the DFEE percentage change values, most likely due to the small data available for DFEE students. However, given recent policy changes, the highest priority is to predict IFEE students, so this variance in DFEE is not of great importance.

Table 3: EFTSL Percentage Change

(a) Average Annual Percentage Change
Faculty of Teaching CSP DFEE IFEE
Business School -0.7276 0.7144 2.0845
Faculty of Arts and Social Sciences -1.9400 -4.9496 6.8222
Faculty of Engineering 1.5220 -2.7069 4.4429
Faculty of Medicine and Health 0.1768 0.2945 0.7948
Faculty of Science -0.6972 -6.1094 5.8540
Sydney Conservatorium of Music 1.8246 -6.5335 3.9446
Sydney Law School -1.2799 -4.5235 4.7425
Sydney School of Architecture, Design and Planning 2.4782 -0.5129 5.8456
(b) Average Annual Forecasted Percentage Change
Faculty of Teaching CSP DFEE IFEE
Business School -0.9877 2.1725 8.4459
Faculty of Arts and Social Sciences -2.3069 -4.5317 6.5422
Faculty of Engineering 1.2167 -3.0673 4.3133
Faculty of Medicine and Health -0.0190 0.2968 0.6242
Faculty of Science -0.6634 -5.5898 6.0633
Sydney Conservatorium of Music 1.5412 -5.8873 3.9948
Sydney Law School -1.0863 -4.1787 5.2968
Sydney School of Architecture, Design and Planning 2.5530 -1.1530 5.7149
(c) Average Annual Estimated Percentage Change
Faculty of Teaching CSP DFEE IFEE
Business School -1.0763 1.3769 1.8731
Faculty of Arts and Social Sciences -1.7139 -2.7046 6.8425
Faculty of Engineering 1.2508 -1.9878 4.0756
Faculty of Medicine and Health 0.2978 1.1331 -0.0387
Faculty of Science -0.3787 -5.3869 5.3839
Sydney Conservatorium of Music 2.2046 -4.7458 4.6665
Sydney Law School -0.8340 -3.6006 4.2388
Sydney School of Architecture, Design and Planning 2.2230 1.1343 5.5214

Comparison to Existing Forecast

In addition to comparing percentage change across the linear model and existing forecast, the two calculated values can be compared graphically for each Faculty, as seen below in Figures 21-28.

Visually, the estimated values appear relatively similar across both models, with the exception of EFTSL for IFEE Business School in 2019-2021. The estimates of the Linear Model are more consistent than the Existing Forecast model, which is intuitive given its linear nature, while the existing forecast model may take in account more external factors which would explain more unpredictable trends in estimate values.

Figure 21: Line Graph of Forecast for Business School across Fee Types.


Figure 22: Line Graph of Forecast for Faculty of Arts and Social Sciences across Fee Types.


Figure 23: Line Graph of Forecast for Faculty of Engineering across Fee Types.


Figure 24: Line Graph of Forecast for Faculty of Medicine and Health across Fee Types.


Figure 25: Line Graph of Forecast for Faculty of Science across Fee Types.


Figure 26: Line Graph of Forecast for Sydney Conservatorium of Music across Fee Types.


Figure 27: Line Graph of Forecast for Sydney Law School across Fee Types.


Figure 28: Line Graph of Forecast for Sydney School of Architecture, Design and Planning across Fee Types.


Budget

Regarding the client’s concerns about meeting budget, the linear model forecasts an overall income of $2,416,435,480 across the major faculties of the University of Sydney in 2024. The split of this income across Faculties can be seen in Table 4 below.

This exceeds the budgets of the previous years, supporting the University’s on-track meeting of the budget for 2024.

Table 4: 2024 Estimated EFTSL and Total Fees for each Faculty
Faculty of Teaching EFTSL Income (AUD$)
Business School 11,581.559 545,770,788
Faculty of Arts and Social Sciences 14,949.297 571,100,838
Faculty of Engineering 8,516.052 378,804,089
Faculty of Medicine and Health 9,859.530 383,259,225
Faculty of Science 8,033.494 311,997,256
Sydney Conservatorium of Music 1,229.235 33,754,546
Sydney Law School 2,053.922 74,482,452
Sydney School of Architecture, Design and Planning 3,133.096 117,266,286
Total 59,356.185 2,416,435,480

Conclusion

By constructing a linear model, EFTSL forecasting within each major Faculty of the University of Sydney has become possible. While this model does not perform as well as the existing forecast method, it has the potential for significant improvement with additional hisotrical data for training and consideration of factors influencing student enrolment that were not available in the initial dataset. With these enhancements, the model can further aid in forecasting enrolments from International Students in light of recent policy changes. However, it is important to consider such policies will lead to new behaviours in student enrolment and the current model is based on old behaviours so there may be further changes unaccounted for by the model. Nevertheless, based on the results of the linear model, the University of Sydney is on track to meet its budget for 2024.

Appendix

A. Fees per EFTSL in each Faculty

Table 5: Fee ($AUD) per EFTSL for each Faculty
Faculty of teaching CSP DFEE IFEE
Faculty of Arts and Social Sciences 18,753.57 32,546.67 50,759.61
Business School 17,631.13 42,939.24 54,844.02
Faculty of Engineering 25,713.36 39,625.79 53,942.01
Faculty of Medicine and Health 29,687.77 44,444.43 58,892.39
Faculty of Science 25,905.67 39,182.93 55,778.32
Sydney School of Architecture, Design and Planning 23,867.44 30,095.07 47,220.94
Sydney Law School 17,239.20 41,322.73 55,219.47
Sydney Conservatorium of Music 23,576.27 33,635.10 48,126.68

B. Coefficients for each Linear model

Table 6: Coefficients Table
Faculty of Teaching Fee Type Intercept Year Semester 2 COVID-19 Impact
Business School CSP 35535 -17 -130 103
Business School DFEE -3786 2 12 5
Business School IFEE -225383 114 216 246
Faculty of Arts and Social Sciences CSP 235613 -115 -49 141
Faculty of Arts and Social Sciences DFEE 20183 -10 -5 29
Faculty of Arts and Social Sciences IFEE -759203 377 297 38
Faculty of Engineering CSP -80252 40 -58 4
Faculty of Engineering DFEE 6288 -3 -7 16
Faculty of Engineering IFEE -345143 172 89 -34
Faculty of Medicine and Health CSP -49735 26 -45 -67
Faculty of Medicine and Health DFEE -43739 22 -20 22
Faculty of Medicine and Health IFEE -988 1 -11 -30
Faculty of Science CSP 754 1 -226 133
Faculty of Science DFEE 14640 -7 -7 3
Faculty of Science IFEE -268150 133 28 33
Sydney Conservatorium of Music CSP -45054 23 -30 18
Sydney Conservatorium of Music DFEE 3615 -2 -1 -4
Sydney Conservatorium of Music IFEE -13249 7 -3 -14
Sydney Law School CSP 10225 -5 -33 35
Sydney Law School DFEE 25185 -12 -7 12
Sydney Law School IFEE -58010 29 1 -18
Sydney School of Architecture, Design and Planning CSP -50922 25 -5 35
Sydney School of Architecture, Design and Planning DFEE -2205 1 -5 8
Sydney School of Architecture, Design and Planning IFEE -141809 71 18 -20

C. Diagnostic Plots for each Linear Model

CSP

Figure 29: Diagnostic Plots for CSP.


DFEE

Figure 30: Diagnostic Plots for DFEE.


IFEE

Figure 31: Diagnostic Plots for IFEE.


CSP

Figure 32: Diagnostic Plots for CSP.


DFEE

Figure 33: Diagnostic Plots for DFEE.


IFEE

Figure 34: Diagnostic Plots for IFEE.


CSP

Figure 35: Diagnostic Plots for CSP.


DFEE

Figure 36: Diagnostic Plots for DFEE.


IFEE

Figure 37: Diagnostic Plots for IFEE.


CSP

Figure 38: Diagnostic Plots for CSP.


DFEE

Figure 39: Diagnostic Plots for DFEE.


IFEE

Figure 40: Diagnostic Plots for IFEE.


CSP

Figure 41: Diagnostic Plots for CSP.


DFEE

Figure 42: Diagnostic Plots for DFEE.


IFEE

Figure 43: Diagnostic Plots for IFEE.


CSP

Figure 44: Diagnostic Plots for CSP.


DFEE

Figure 45: Diagnostic Plots for DFEE.


IFEE

Figure 46: Diagnostic Plots for IFEE.


CSP

Figure 47: Diagnostic Plots for CSP.


DFEE

Figure 48: Diagnostic Plots for DFEE.


IFEE

Figure 49: Diagnostic Plots for IFEE.


CSP

Figure 50: Diagnostic Plots for CSP.


DFEE

Figure 51: Diagnostic Plots for DFEE.


IFEE

Figure 52: Diagnostic Plots for IFEE.